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Large numbers of U.S. students lack proficiency in science, 
and students from different racial/ethnic and socioeconomic 
groups show disparities in science achievement. Science 
knowledge and skills are important for both academic and 
workplace success, and a variety of interventions have been 
developed to improve student achievement in science. 
These interventions include professional development 
programs for teachers that are designed to change teachers’ 
content knowledge or classroom instruction in science. 
Science Teachers Learning through Lesson Analysis (STeLLA’) 
is a professional development program, developed by BSCS 
Science Learning, that aims to improve students’ science 
achievement by improving teachers’ science content 
knowledge and their abilities to (a) explain science concepts 
to students, (b) clearly identify to students the science 
concepts used in student learning activities, and (c) engage 
students in thinking about science. 


Participating teachers learn from postsecondary science faculty 
to improve their knowledge of the science content areas they 
will teach during the school year; become familiar with STeLLA® 
lesson plans and instructional strategies; work in study groups 
led by STeLLA® professional development leaders to analyze 
videorecorded examples of experienced teachers implementing 
STeLLA’ lesson plans and strategies; and videorecord their 

own classroom instruction and meet with their study group to 
analyze one another’s videorecorded lessons. 


This What Works Clearinghouse (WWC) intervention report, part 
of the WWC’s Primary Science topic area, explores the effects 
of STeLLA® on science achievement. The WWC identified three 
studies of STeLLA’. One of these studies meets WWC standards. 
The evidence presented in this report is from one study of the 
effects of STeLLA® on students—including 59% White, 4% Black, 
2% Asian, and 28% Hispanic students—in grades 4 and 5 in 
urban, suburban, and rural schools in Colorado. 


What Happens When Students Participate in STeLLA*? 


The evidence indicates that implementing STeLLA® has 
potentially positive effects on science achievement 
compared with another yearlong teacher professional 
development program. 


Findings on STeLLA® from the one study that meets WWC 
standards are shown in Table 1. The table reports an 
effectiveness rating, an improvement index, and the number 
of studies and students that contributed to the findings. The 
effectiveness rating is based on the quality of the designs 
used in studies, whether the findings are favorable or 
unfavorable for the intervention, and the number of studies 
that tested the intervention. See Box 1 for more information 
on interpreting effectiveness ratings. 


In order to help readers judge the practical importance of 
an intervention’s effect, the WWC translates findings across 


studies into an “improvement index” by averaging findings 
that meet WWC standards within the same outcome domain. 
The improvement index can be interpreted as the expected 
change in percentile rank for an average comparison group 
student if that student had received the intervention. For 
example, an improvement index of +25 means that the 
expected percentile rank of the average comparison group 
student would increase by 25 points if the student received 
instruction from a teacher who had participated in STeLLA” 
instead of the professional development program provided 
to the comparison group teachers. A positive improvement 
index does not necessarily mean the estimated effect is 
statistically significant. 


The evidence presented in this report is based on available 
research. Findings and conclusions could change as new 
research becomes available. 


Table 1. Summary of findings on STeLLA’ from one study that meets WWC standards 


Study Findings Evidence meeting WWC standards (version 4.0) 


Improvement index 


Outcome domain Effectiveness rating (percentile points) Number of studies Number of students 


Science achievement Potentially positive effects +25 1 2,823 


Note: The effects of STeLLA® are not known for other outcomes within the Primary Science topic area, including life sciences, physical sciences, and earth/space sciences. 


BOX 1. HOW THE WWC REVIEWS AND DESCRIBES EVIDENCE 


The WWC evaluates evidence based on the quality and results of reviewed studies. The criteria the WWC uses for evaluating 
evidence are defined in the Procedures and Standards Handbooks and the Review Protocols. The studies summarized in this report 
were reviewed under WWC Standards (version 4.0) and the Primary Science topic area protocol (version 4.0). 


To determine the effectiveness rating, the WWC considers what methods each study used, the direction of the effects, and the 
number of studies that tested the intervention. The higher the effectiveness rating, the more certain the WWC is about the reported 
results and about what will happen if the same intervention is implemented again. The following key explains the relationship between 
effectiveness ratings and the statements used in this report: 


Effectiveness Rating 


Rating interpretation 


Description of the evidence 


Positive (or negative) effects 
outcome 


The intervention is likely to change an 


Strong evidence of a positive (or negative) 
effect, with no overriding contrary evidence 


Potentially positive (or negative) effects 


The intervention may change an outcome 


Evidence of a positive (or negative) effect with 
no overriding contrary evidence 


No discernible effects 


The intervention may result in little to no 
change in an outcome 


No affirmative evidence of effects 


Mixed effects 
on an outcome 


How is STeLLA’ Implemented? 


The following section provides details of how districts and 
schools can implement STeLLA’. This information can help 
educators identify the requirements for implementing 
STeLLA’® and determine whether implementing this interven- 
tion would be feasible in their districts or schools. Informa- 
tion on STeLLA’ presented in this section comes from the 
study that meets WWC standards (Taylor et al., 2017) and 
from correspondence with the developer. 


¢ Goal: STeLLA’ aims to improve student science achieve- 
ment by improving a teacher’s ability to (a) explain sci- 
ence concepts to students, (b) clearly identify to students 
the science concepts used in student learning activities, 
and (c) engage students in thinking about science. 


¢ Target population: STeLLA’ is offered to teachers who 
provide science instruction to students in kindergarten 
to grade 12. Training is also available for school or district 
personnel who wish to become STeLLA’ professional 
development leaders. 


¢ Method of delivery: Teachers learn from postsecondary 
science faculty and STeLLA’ professional development 
leaders at an in-person summer institute. During the 
institute, teachers begin improving their science content 
knowledge, become familiar with STeLLA’ lesson plans 
and instructional strategies, and learn to analyze videos 
of experienced teachers delivering model lesson plans 
and using the instructional strategies. During the school 
year, teachers videorecord themselves delivering model 
STeLLA’ lesson plans and meet in study groups at their 
school with in-person or online guidance from their 
STeLLA’ professional development leader to analyze their 
own and one another’s videorecorded lessons. 


The intervention has inconsistent effects 


Evidence includes studies in at least two of 
these categories: studies with positive effects, 
studies with negative effects, or more studies 
with indeterminate effects than with positive or 
negative effects 


Comparison condition: In the one study that con- 
tributes to this intervention report, students in the 
comparison group received science instruction from 
teachers who participated in a yearlong professional 


development program designed to improve only their 
science content knowledge. These teachers did not 
receive STeLLA’ materials or training and support from 
STeLLA’ professional development leaders. 


Frequency and duration of service: STeLLA’ is a year- 
long professional development program for teachers that 
begins with a 2-week summer institute (60 hours) fol- 
lowed by eight monthly grade-level study group meetings 
during the school year (30 hours). Personnel who wish to 
become STeLLA’ professional development leaders require 
an additional 1 to 2 years of training. Refer to Table 2 for 
additional details. 


Intervention components: The key components of 
STeLLA® are described in Table 2. Training for personnel 
to become STeLLA’ professional development leaders, 
which can be bundled with the teacher program, is also 
described in Table 2 but was not part of the study that 
contributes to this intervention report. 


Table 2. Components of STeLLA® 


Key component 


Description 


Teacher 
professional 
development 


STeLLA® lesson 
plans 


STeLLA® teaching 
strategies 


Analysis of 
videorecorded 
lessons in study 
groups 


STeLLA® 
leadership 
development 
training 


During the yearlong STeLLA® professional development program, teachers (a) develop science content knowledge of two topics (for example, 
“the earth’s changing surface” and “matter and molecules in the water cycle”) that they will teach in their own classrooms in the upcoming 
school year, (b) review and implement STeLLA® model lesson plans, and (c) learn instructional strategies to support student learning. 


The professional development program begins with a 2-week (58 to 60 hours) summer institute facilitated by postsecondary science 
faculty and a STeLLA® professional development leader. Science faculty engage teachers in short lectures, readings, activities, and 
discussions of science concepts. STeLLA® professional development leaders lead teachers through a review of content-specific 
background documents that describe how science lesson activities, representations of science concepts, and analogies that are 
typically used in science instruction might support or interfere with student learning. STeLLA® professional development leaders also 
introduce teachers to the STeLLA® lesson plans, teaching strategies, and process for analyzing videorecorded lessons. 


During the school year, teachers meet with school-based study groups of other teachers of students in the same grade and their 
STeLLA® professional development leader monthly for eight discussion sessions each lasting 3 to 4 hours to continue learning how 
to convey science content to students and how to implement STeLLA® teaching strategies in their own classrooms. 


At the summer institute, participating teachers receive STeLLA® model lesson plans for one of the two topics. The model lesson 
plans are detailed guides designed to build teachers’ science content knowledge and familiarity with STeLLA® teaching strategies. 
They highlight common student responses to teacher questions and suggest ways for teachers to respond to students’ ideas, 
predictions, and questions. Teachers review these lesson plans and analyze videos of experienced teachers delivering them. In the 
fall, teachers practice implementing the model lesson plans in their own classrooms. 


Teachers learn instructional strategies that emphasize: 


* Explaining science concepts, including using accurate descriptions, analogies, visual displays or models, and other 
representations of scientific content that minimize student misconceptions; allowing students to practice using these 
representations; describing the connections between science concepts; and summarizing key elements of these concepts. 

* Clearly identifying to students the science concepts used in student learning activities, including establishing a learning 
goal with a central question or objective, selecting activities matched to the objective, sequencing science content and learning 
activities appropriately, and identifying how the learning activities illustrate or draw on science concepts. 

¢ Engaging students in thinking about science, including asking questions to elicit and challenge student ideas and predictions; 
engaging students in interpreting data, summarizing science concepts, and applying those concepts to new contexts or 
phenomena; and encouraging students to communicate in scientific ways. 


Teachers first become familiar with STeLLA® strategies at the summer institute by watching videos of experienced teachers using them in 
science lessons. Teachers later practice using the strategies in planning and delivering science instruction in their own classrooms. 


ASTeLLA? professional development leader works with school-based study groups to analyze videorecorded examples of science instruction and 
accompanying examples of student written work. Teachers discuss their own and students’ understanding of the science topics covered in the 
lessons and learn to identify STeLLA® instructional strategies that the videorecorded teacher used or could have used to support student learning. 


At the summer institute, teachers begin learning the lesson analysis process by discussing videorecorded lessons of experienced 
teachers implementing STeLLA® model lesson plans. During the school year, teachers videorecord themselves teaching the two science 
topics in their own classrooms and then analyze one another's videorecorded lessons and accompanying examples of student work. 

In the first half of the school year, teachers analyze videos of one another delivering STeLLA® model lesson plans. In the second half 

of the school year, teachers collaborate with their study group to develop their own science lesson plans for the second science topic 

by incorporating STeLLA® teaching strategies and anticipating student misconceptions. Teachers then deliver and videorecord these 
lessons and meet once more to analyze one another's instruction and summarize what they have learned over the yearlong program. 


Also available is a 2- to 3-year program for school or district personnel to become STeLLA® professional development leaders. 
Participants complete 1 year of leadership training before they begin providing STeLLA® professional development to a cohort 
of teachers. During the second year of the program, the leaders-in-training receive support from the STeLLA® developer. This 
component was not part of the study that contributes to this intervention report. 


What Does STeLLA° Cost? 


This preliminary list of costs is not designed to be exhaustive; rather, it provides educators an overview of the major resources 
needed to implement STeLLA®”. The program costs described in Table 3 are based on the information available as of May 2020. 


Table 3. Cost ingredients for STeLLA° 


Cost ingredients Description Source of funding 


Personnel 


Implementing STeLLA® costs approximately $1,750 per teacher for the yearlong program. Schools and districts pay BSCS Science 
Each participating teacher attends a 2-week summer institute and eight monthly study group —_ Learning for costs of providing the STeLLA® 
sessions (up to 4 hours each) during the school year. A 2-year leadership development summer institutes and support for partic- 
program for school or district personnel to become STeLLA® professional development __ipating teachers (or leaders-in-training) 
leaders costs approximately $10,000 per leader, but this component is not required. throughout the following school year. 


Cost ingredients Description 


Source of funding 


The 2-week summer institute typically takes place at a facility provided by the school 
district. School-year study group meetings take place at participating schools, with the 
professional development leader joining online or on-site depending on the availability 
of local STeLLA® professional development leaders and the agreement with BSCS 
Science Learning. 


Schools and districts are responsible for 
providing the facility for the summer institute 
and the transportation to and from the facility. 
Schools and districts provide physical space 
for teacher professional development and 
classroom instruction during the school year. 


Teacher participants receive a set of grade-appropriate STeLLA® lesson plans for one of the 
two topics they will teach in the upcoming year, transcripts of program-developed videorecorded 
lessons and related examples of student work, a STeLLA® strategy guide, lesson analysis 
protocols, and other resources associated with STeLLA®. BSCS Science Learning provides 
teachers with online access to videorecordings made by members of their local study group. 


Participants training to become STeLLA® professional development leaders receive the same 
materials provided to teacher participants as well as videorecordings and transcripts from 
STeLLA® professional development sessions with teachers and additional resources specific 
to the leadership development program. 


For More Information: 


About STeLLA® 
BSCS Science Learning, 5415 Mark Dabling Boulevard, Colorado Springs, CO 80918 
Attn: Jody Bintz, Associate Director for Strategic Partnerships & Professional Learning 


The fees that school or districts pay for 
teacher (or leader-in-training) professional 
development cover the cost of program 
materials provided to participants. Schools 
and districts are responsible for the costs 
of videorecording participating teachers. 


Email: jbintz2@bscs.org Web: https://oscs.org/our-work/rd-programs/stella-science-teachers-learning-from-lesson-analysis/. 


Phone: (719)-513-5550 


About the cost of the intervention 
Actual program costs available in consultation with the developer. 


About the study that meets WWC standards 


Taylor, J., Roth, K., Wilson, C., Stuhlsatz, M., & Tipton, E. (2017). The effect of an analysis-of-practice, videocase-based, 
teacher professional development program on elementary students’ science achievement. Journal of Research on Educational 


Effectiveness, 10(2), 241-271. https://eric.ed.govw/?id=EJ1135795 


In What Context Was STeLLA’ Studied? 


The following section provides information on the setting 
of the one study of STeLLA® that meets WWC standards, 
and a description of the participants in the research. This 


WHERE THE STUDY WAS CONDUCTED 


information can help educators understand the context in 
which the study of STeLLA’ was conducted and determine 
whether the program might be suitable for their setting.a 


2,823 students in 77 elementary schools in Colorado 


Ethnicity 


4% 2% 


Ea 


White Black Asian Other/Not specified Hispanic 


Free & Reduced-Price Lunch: 43% — English Learners: 12% 


Grades 
Grade 4—5 


LEARN MORE 


Urban, suburban, and rural districts 


Non-Hispanic 


Gender: 48% Female 52% Male 


Read more about the STeLLA® intervention and the study that is summarized in this brief in the Intervention Report. 


